Standard Deviated Group

Group members:

Edoardo Cocciò - 282401
Alexandra Tabarani - 282091
Marta Torella - 284091
Simone Filosofi - 284531

Task 1

The dataset we are going to analyze is the “Telecom Churn” dataset, which contains information about US Telecom. For this first task, our objective is to predict customer churn, enabling the company to implement preventative measures to retain clients; as a matter of fact, by identifying customers at high risk of churn, the company can proactively offer targeted promotions to improve retention rates.

EDA

We start by performing an exploratory data analysis (EDA) to understand the structure of the dataset and the relationships between the variables. Starting with a general overview of the dataset, we can see that it contains 3333 observations and 20 variables. The variables are a mix of numerical and categorical variable; we proceed by factoring the categorical variables in order to better deal with them later.

##  [1] "State"                  "Account.length"         "Area.code"             
##  [4] "International.plan"     "Voice.mail.plan"        "Number.vmail.messages" 
##  [7] "Total.day.minutes"      "Total.day.calls"        "Total.day.charge"      
## [10] "Total.eve.minutes"      "Total.eve.calls"        "Total.eve.charge"      
## [13] "Total.night.minutes"    "Total.night.calls"      "Total.night.charge"    
## [16] "Total.intl.minutes"     "Total.intl.calls"       "Total.intl.charge"     
## [19] "Customer.service.calls" "Churn"

Missing Values and Duplicates

A crucial step in the data analysis process is to check for missing values and duplicates in the dataset to ensure that the data is clean and ready for analysis; in our case we have no missing values and no duplicates. After ensuring dataset integrity, we proceed with the analysis.

Univariate Analysis

We proceed by conducting an univariate analysis of the variables in our dataset: our first step is to check the distribution of the target variable ‘Churn’; indeed, we identified the variable ‘Churn’ as the target of our analysis since, as previously stated, the goal of our first task is to build a model to predict customer churn. As shown by the below pie chart,the distribution of the target variable is unbalanced, with 85.5% of the customers not churning and 14.5% churning.

We now proceed with the univariate analysis of the numerical variables in the dataset. We will plot the distribution of the numerical variables by the target variable ‘Churn’ to see if there are any differences in the distribution of the numerical variables between the two levels of the target variable.

The above univariate analysis of the numerical variables shows that the variables distribution differs remarkably for the two levels of the target variable “Churn” in a couple of cases. Indeed, there’s a noticeable difference in the distribution of customer service calls: customers who churned seem to make more customer service calls, which could indicate a relationship between customer satisfaction and churn. It also appears that customers who churned tend to have slightly higher total day minutes and total day charges; this might suggest that customers with higher usage during the day are more likely to churn. The other variables do not show a significant difference in distribution between the two levels of the target variable.

Another remarkable imbalance can be found in the distribution of both International.plan and Voice.mail.plan. The majority of customers do not have an international plan nor a voice mail plan; this might suggest that these services are not very popular among customers.

To conclude the analysis of the categorical variables, we also analysed the distribution of the State variable; the below plot shows the distribution of customers across different states in the US.

us_map <- map_data("state") %>% 
  as_tibble()

us_map %>%
  ggplot(aes(long, lat, map_id = region)) + 
  geom_map(
    map =us_map,
    color = "gray80",
    fill = "gray30",
    size = 0.3
  ) + 
  coord_map("ortho", orientation = c(39, -98, 0)) + 
  labs(
    title = "US Map",
    subtitle = "Based on Latitude and Longitude"
  ) + theme(plot.title = element_text(hjust=0.5))

state_names <- c(KS = "kansas", OH = "ohio", NJ = "new jersey", OK = "oklahoma",
                 AL = "alabama", MA = "massachusetts", MO = "missouri", LA = "louisiana",
                 WV = "west virginia", IN = "indiana", RI = "rhode island", IA = "iowa",
                 MT = "montana", NY = "new york", ID = "idaho", VT = "vermont",
                 VA = "virginia", TX = "texas", FL = "florida", CO = "colorado",
                 AZ = "arizona", SC = "south carolina", NE = "nebraska", WY = "wyoming",
                 HI = "hawaii", IL = "illinois", NH = "new hampshire", GA = "georgia",
                 AK = "alaska", MD = "maryland", AR = "arkansas", WI = "wisconsin",
                 OR = "oregon", MI = "michigan", DE = "delaware", UT = "utah",
                 CA = "california", MN = "minnesota", SD = "south dakota", NC = "north carolina",
                 WA = "washington", NM = "new mexico", NV = "nevada", DC = "district of columbia",
                 KY = "kentucky", ME = "maine", MS = "mississippi", TN = "tennessee",
                 PA = "pennsylvania", CT = "connecticut", ND = "north dakota")

churn.tbl <- data %>% select(State, Churn) %>%
  group_by(State) %>%
  summarize(
    total_customers = n(),
    churned_customers = sum(Churn == "True"),
    churn_rate = (churned_customers / total_customers),
    churn_rate_txt = scales::percent(churn_rate)
  ) %>%
  mutate(full_state = state_names[as.character(State)]) %>%
  ungroup() %>% 
  left_join(us_map, by=c("full_state" = "region"))

us_map_sf <- st_as_sf(us_map, coords = c("long", "lat"), crs = 4326, agr = "constant")

centroids <- us_map_sf %>%
  group_by(region) %>%
  summarise(geometry = st_centroid(st_union(geometry)))

centroids_df <- as.data.frame(st_coordinates(centroids))
centroids_df$region <- centroids$region

churn.tbl <- churn.tbl %>%
  left_join(centroids_df, by = c("full_state" = "region"))

churn.tbl %>%
  ggplot(aes(long, lat)) + 
  geom_map(
    aes(map_id = full_state),
    map = us_map,
    color = "gray86",
    fill = "gray30",
    size = 0.3
  ) + coord_map("ortho", orientation = c(39, -98, 0)) + geom_polygon(aes(group = group, fill = churn_rate), color="black") +
  scale_fill_gradient2("",low = "#18BC9C", mid = "white", high = "#E31A1C", midpoint = 0.10, labels = scales::percent) + 
  geom_text(aes(x = X, y = Y, label = State), stat = "unique", size = 3, inherit.aes = FALSE, fontface = "bold") +
  theme_void() + 
  theme(
    plot.title = element_text(size=18, face="bold", color = "#2C3E50"),
    legend.position = "right"
  ) + 
  labs(
    title = "Churn Rate Across US States",
    x = "",
    y = ""
  ) + 
  theme(
    plot.title = element_text(hjust = 0.5, face = "bold"),
  )

Hypothesis testing

Correlation analysis

The correlation matrix shows that there is a perfect correlation between the charge variables and minutes variables so we can drop all the minutes one we drop Total.day.minutes, Total.eve.minutes, Total.night.minutes, Total.intl.minutes.

Lower-dimensional models

Before proceeding with the full model we need to focus on some lower dimensional model in order to investigate some interesting relationships between the variables. We only include customers who have made a service call:

service_calls <- data %>%
  filter(Customer.service.calls > 0)

This is the proportion of customers who churned after making a service call:

We can see that the churn rate increases as the number of customer service calls increases. This suggests that customers who make more service calls are more likely to churn.

(decidere se gli vogliamo mettere tutti o meno)

N.B. We computed more analysis, using plots and hypothesis testing, to understand the relationship between the variables and the target variable “Churn”, but for limited space restrictions we will not show them here. In the above paragraph (‘Conclusions up to now’) we will report the main conclusions arising from this analysis.

CONCLUSION UP TO NOW:

we dropped the minutes variables because they are perfectly correlated with the charge variables;
we saw that the churn rate is higher for customers that have an international plan;
we saw that the churn rate is higher for customers that do not have a voice mail plan;
we saw that the churn rate is higher for customers that have a higher number of customer service calls;
we saw that the churn rate is higher for customers that have higher total day minutes and total day charge;
we saw that the churn rate is higher for customers that have higher total eve charge, night charge, and intl charge;
we saw that State also influences the churn rate;

Preprocessing and feature engineering

We should first deal with the 51 states: we can group them by region (north, south, east, west).

We’ll use the regions column for the first models to avoid having high-cardinality encoded columns: we store the variable in order to use it later with more robust models (tree-based).

state.column <- data$State
data$State <- NULL

Now let’s re-do the chi square:

contingency.table <- table(data$Region, data$Churn)
chisq.test(contingency.table)

## 
##  Pearson's Chi-squared test
## 
## data:  contingency.table
## X-squared = 5.9403, df = 4, p-value = 0.2036

H0: Region and Churn are independent, H1: Region and Churn are dependent. Since p-value > 0.05, we fail to reject the null hypothesis, therefore the region does not influence the churn rate. Let’s see the distribution of churn by region:

We can see that the churn rate is not influenced by the region.

Splitting the data and scaling

set.seed(1)
ids.train <- sample(1:nrow(data), size = 0.75 * nrow(data), replace = FALSE)
data.train <- data[ids.train,]
data.val <- data[-ids.train,]

We oversample the minority class to balance the dataset, now we have 4260 total observations.

data.train.balanced <- ovun.sample(Churn ~ ., data = data.train, method = "over", N=4260)$data
table(data.train.balanced$Churn)

## 
## False  True 
##  2130  2130

In order to to prevent certain columns from dominating the model, we scale it using the training data’s mean and standard deviation:

cols_to_scale <- c("Account.length", "Area.code", "Number.vmail.messages", "Total.day.calls", "Total.day.charge", "Total.eve.calls", "Total.eve.charge", "Total.night.calls", "Total.night.charge", "Total.intl.calls", "Total.intl.charge", "Customer.service.calls")

train.mean <- apply(data.train.balanced[, cols_to_scale], MARGIN = 2, FUN = mean)
train.sd <- apply(data.train.balanced[, cols_to_scale], MARGIN = 2, FUN = sd)

data.train.balanced[, cols_to_scale] <- scale(data.train.balanced[, cols_to_scale], center = train.mean, scale = train.sd)
data.val[, cols_to_scale] <- scale(data.val[, cols_to_scale], center = train.mean, scale = train.sd)

Baseline Logistic Regression Model

Our starting point is a logistic regression model. We will use this model as a baseline to compare the performance of more complex models:

logistic.baseline <- glm(Churn ~ ., data = data.train.balanced, family = "binomial")

The baseline model shows decent accuracy at 83.33%, however, precision at 57.89% and recall at 42.04% indicate areas for potential improvement. This is normal since we still have to perform feature selection and regularization to improve the model.

##              
## baseline.pred False True
##             0   629   48
##             1    91   66

##    Accuracy Precision    Recall        F1
## 1 0.8333333 0.5789474 0.4203822 0.4870849

AIC and BIC Model Selection

We performed the Akaike Information Criterion (AIC) stepwise selection to identify the most important predictors in the model:

akaike.fw <- step(glm(Churn ~ 1, family = "binomial", data = data.train.balanced), scope = formula(logistic.baseline), direction = "forward")
akaike.back <- step(logistic.baseline, direction = "backward")
akaike.both <- step(logistic.baseline, direction = "both")

Across all selection methods, certain predictors consistently appear as significant: “International.plan”, “Customer.service.calls”, “Total.day.charge” and “Voice.mail.plan”.

Bayesian Information criterion (BIC) is another criterion to select the best model. We will use the BIC to compare the models selected by the AIC:

bayesian.fw <- step(glm(Churn ~ 1, family = "binomial", data = data.train.balanced), scope = formula(logistic.baseline), direction = "forward", k = log(nrow(data.train.balanced)))
bayesian.back <- step(logistic.baseline, direction = "backward", k = log(nrow(data.train.balanced)))
bayesian.both <- step(logistic.baseline, direction = "both", k = log(nrow(data.train.balanced)))

Considering bidirectional elimination of both methods, BIC is more strict as it removes all the covariates representing the regions.

After that we compare the validation set results:

The results show that the model selected by the AIC method performs slightly better than the one selected by the BIC method. The AIC model has a higher accuracy, precision, recall, and F1 score: maybe dropping all the region features as BIC does is too big of a loss of informations.

AIC:

##             
## akaike.preds False True
##            0   631   47
##            1    89   67

##    Accuracy Precision    Recall        F1
## 1 0.8369305 0.5877193 0.4294872 0.4962963

BIC:

##               
## bayesian.preds False True
##              0   627   48
##              1    93   66

##    Accuracy Precision    Recall        F1
## 1 0.8309353 0.5789474 0.4150943 0.4835165

ROC Curves

AUC: In order to insight into the models’ ability to evaluate their predictive performance in discerning churn, we’re calculating the Area Under the Curve (AUC) for Akaike, Bayesian, and Baseline models.

auc_akaike <- as.numeric(auc(roc(data.val$Churn, predict(akaike.both, newdata = data.val, type = "response"))))
auc_bayesian <- as.numeric(auc(roc(data.val$Churn, predict(bayesian.both, newdata = data.val, type = "response"))))
auc_full <- as.numeric(auc(roc(data.val$Churn, baseline.pred)))

LASSO and RIDGE REGULARIZATION

We will use LASSO and Ridge regularization to improve the logistic regression model. We will use:

-glmnet package to perform the regularization;

cv.glmnet function to perform cross-validation to find the optimal value of the regularization parameter lambda;
glmnet function to fit the model with the optimal lambda value.

LASSO:

set.seed(1)
ctrl <- trainControl(method = "cv", number = 10)
lasso <- train(Churn ~ ., data = data.train.balanced, method = "glmnet", metric = "Accuracy", trControl = ctrl, tuneGrid = expand.grid(alpha = 1, lambda = seq(0, 0.15, length = 30)))
max(lasso$results$Accuracy)

## [1] 0.7741784

lasso$bestTune

##   alpha lambda
## 1     1      0

lasso.predict <- predict(lasso, data.val)
lasso.cm <- table(lasso.predict, data.val$Churn)
lasso.metrics <- get.metrics(lasso.cm)

RIDGE:

set.seed(1)
ridge <- train(Churn ~ ., data = data.train.balanced, method = "glmnet", metric = "Accuracy", trControl = ctrl, tuneGrid = expand.grid(alpha = 0, lambda = seq(0, 0.15, length = 30)))
max(ridge$results$Accuracy)

## [1] 0.7706573

ridge$bestTune

##   alpha     lambda
## 3     0 0.01034483

ridge.predict <- predict(ridge, data.val)
ridge.cm <- table(ridge.predict, data.val$Churn)
ridge.metrics <- get.metrics(ridge.cm)

Now let’s compare the metrics obtained with the different models:

##            Model  Accuracy Precision    Recall  F1_Score       AUC
## 1         Akaike 0.8369305 0.5877193 0.4294872 0.4962963 0.8249269
## 2       Bayesian 0.8309353 0.5789474 0.4150943 0.4835165 0.8343567
## 3  Full Logistic 0.8333333 0.5789474 0.4203822 0.4870849 0.7262792
## 4 Logistic Ridge 0.7853717 0.7631579 0.3640167 0.4929178 0.7760234
## 5 Logistic Lasso 0.7889688 0.7631579 0.3686441 0.4971429 0.7781067

Our baseline logistic regression model demonstrates moderate performance across various metrics:

following analyses under different approaches, accuracy ranges from 78.54% to 83.69%;
precision consistently hovers around 76%, indicating a high percentage of correctly predicted positive cases;
recall is lower, ranging from 36% to 43%, suggesting room for improvement in capturing all positive cases;
the F1 score, a balanced metric, varies from 0.49 to 0.50. AUC values, indicating model discrimination, range from 0.73 to 0.83;

Overall, the model performs reasonably well but could benefit from increased recall to capture more positive cases.

Other models

From now on we are studying other models and comparing the metrics with the logistic regression we chose as baseline model:

Decision trees
Random Forest
Gradient Boosting

Decision Trees

First of all we split again to undo the scaling and to make trees more interpretable for us:

set.seed(1)
ids.train <- sample(1:nrow(data), size = 0.75 * nrow(data), replace = FALSE)
data.train <- data[ids.train,]
data.val <- data[-ids.train,]

Then we oversampled the minority class in order to have a total of 4260 observations as in the previous case:

data.train.balanced <- ovun.sample(Churn ~ ., data = data.train, method = "over", N=4260)$data
table(data.train.balanced$Churn)

##          
## tree.pred False True
##     False   651   14
##     True     69  100

##    Accuracy Precision   Recall        F1
## 1 0.9004796  0.877193 0.591716 0.7067138

Pruning the tree

In order to improve the performance and generalization ability of decision trees (to prevent overfitting) we adopted a Pruning approach with cross-validation:

After pruning, the resuting tree is the following:

The predictive performance of the pruned tree is evaluated on the validation dataset, giving as contingency table the below:

##            
## pruned.pred False True
##       False   651   14
##       True     69  100

Random Forest

Now its the turn of the Random Forest model. Using again a seed set for reproducibility, we train the model on a balanced training dataset, configuring parameters such as the number of trees and variable selection for each split (500).

set.seed(1)
fit.bag <- randomForest(Churn ~ ., data = data.train.balanced, mtry = ncol(data.train.balanced) - 1, ntree = 500, importance = T)

In this section, we evaluate our bagged model’s performance on the validation dataset:

## [1] 0.01296275

## [1] 0.03357314

##                 
## validation.preds False True
##            False   713   21
##            True      7   93

## Setting levels: control = False, case = True

## Setting direction: controls < cases

We utilized two metrics for error estimation: Out-of-Bag (OOB) test error, which is approximately 1.33%, and true test error, which is approximately 3.72%. By comparing these figures, we observe that with a larger number of trees, the OOB test error estimation could potentially approach the true test error more closely. Additionally, we examine the contingency table of predicted versus actual outcomes, which illustrates the model’s performance. For instance, out of 720 instances predicted as ‘False’, 709 are actually ‘False’, while out of 114 instances predicted as ‘True’, 94 are indeed ‘True’.

eXtreme Gradient Boosting

We’re now using XGBoost to build a churn prediction model. First, we’ll prepare our data by encoding categorical variables and splitting it into training and validation sets, then, we’ll train the XGBoost model with specified parameters.

X.tr <- model.matrix(Churn ~ ., data = data.train.balanced)[, -1]
y.tr <- as.numeric(data.train.balanced$Churn)-1
X.val <- model.matrix(Churn ~ ., data = data.val)[, -1]
y.val <- as.numeric(data.val$Churn)-1
fit.xg <- xgboost(as.matrix(X.tr), label = y.tr, nrounds = 50, objective = "binary:logistic", eval_metric = "error")

As shown from the results below, the XGBoost model achieves strong performance with a 97% of accuracy, a recall of 93.20% for ‘Churn’ instances and a precision of 84.21%. It exhibits low rates of false negatives (18 out of 114) and false positives (7 out of 731).

##        
## xg.pred False True
##       0   713   18
##       1     7   96

##   Accuracy Precision    Recall        F1
## 1 0.970024 0.8421053 0.9320388 0.8847926

## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.

Feature Importance

Notably, total day minutes, customer service calls, and the presence of an international plan emerge as the top features, indicating their significant influence on customer churn. Other factors such as evening and international call durations also contribute to the model’s predictive power, albeit to a lesser extent.

## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [00:51:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.

Here we can see from the confusion matrix that the model has a good performance, but if compared to the matrix shown at the beginning of the section, we can see that the model improved in the prediction of true positives (96 before, 98 now), but has a slightly worse performance in predicting true negatives (713 before, 707 now).

##           
## pred_xg_cv False True
##      False   707   16
##      True     13   98

Main conclusions

To sum up, we have performed a comprehensive analysis of the churn dataset, including data exploration, preprocessing, and feature engineering. We have built a logistic regression model as a baseline and compared its performance with other models, such as decision trees, random forests, and gradient boosting.

##               Model  Accuracy Precision    Recall  F1_Score       AUC
## 1            Akaike 0.8369305 0.5877193 0.4294872 0.4962963 0.8249269
## 2          Bayesian 0.8309353 0.5789474 0.4150943 0.4835165 0.8343567
## 3     Full Logistic 0.8333333 0.5789474 0.4203822 0.4870849 0.7262792
## 4    Logistic Ridge 0.7853717 0.7631579 0.3640167 0.4929178 0.7760234
## 5    Logistic Lasso 0.7889688 0.7631579 0.3686441 0.4971429 0.7781067
## 6 CV Decision Trees 0.9004796 0.8771930 0.5917160 0.7067138 0.8906798
## 7    Random Forests 0.9664269 0.8157895 0.9300000 0.8691589 0.9030336
## 8            CV XGB 0.9652278 0.8596491 0.8828829 0.8711111 0.9207968

Now to draw the main conclusions from our analysis let’s compare the metrics obtained with the different models’ final data:

Best classification model: Random Forests demonstrate the best overall performance, with an accuracy of 96.64%, precision of 81.58%, recall of 93.00%, F1-score of 86.92%, and area under the curve (AUC) of 90.30%. This indicates that Random Forests is the most effective model in predicting churn.
Boosting models: CV XGBoost shows an accuracy of 96.52%, precision of 85.96%, recall of 88.29%, F1-score of 87.11%, and AUC of 92.08%, demonstrating performance similar to Random Forests and superior to other classification models.
Linear models: Linear models (Akaike, Bayesian, Full Logistic, Logistic Ridge, Logistic Lasso) exhibit lower performance compared to boosting models and decision trees, with AUC ranging from 72.63% to 83.44%. However, Logistic Ridge and Logistic Lasso show notably higher precision compared to other linear models.
Decision trees: CV Decision Trees demonstrate good performance with an accuracy of 90.05%, precision of 87.72%, recall of 59.17%, F1-score of 70.67%, and AUC of 89.07%, but are surpassed by boosting models and Random Forests in terms of AUC.

In conclusion, Random Forests and CV XGBoost emerge as the most effective models for churn prediction, with Random Forests showing slightly better performance in terms of AUC.

Task 2

For this second task, we will focus on the customer segmentation. We will use clustering techniques (kmeans and hierarchical clustering) to group customers based on their characteristics and behaviour. The goal is to identify different customer segments that can help the company to better understand its customers and tailor its marketing strategies.

Feature Engineering

Before proceeding with the clustering, we need to preprocess the data and engineer some features. Indeed, we drop the columns that are not useful for the clustering analysis, we create three new features that can help to better identify the customer segments, and lastly, we scale the numerical variables.

##   Number.vmail.messages Total.minutes Total.charge Total.calls
## 1             1.2346975     1.3933218   1.53397886 -0.06203854
## 2             1.3077522     0.3705797  -0.01997227  0.77980605
## 3            -0.5916711    -0.5832384   0.27044139  0.80883517
## 4            -0.5916711    -0.3008727   0.69987274 -1.45543647
## 5            -0.5916711    -0.8878377  -0.70077807  1.56359239
## 6            -0.5916711     0.6929658   0.77699899  0.51854394

As shown by the above table, we decided to consider the following features for the clustering analysis: Number.vmail.messages, Total.minutes, Total.charge, Total.calls. These features were chosen to capture a comprehensive view of customer behaviors, focusing on voicemail usage, overall call time, total expenditure, and call frequency. This selection aims to segment the customer base effectively, revealing patterns that can inform targeted marketing strategies, service optimizations, and customer retention efforts. By analyzing both behavioral and financial dimensions, the company can better understand diverse customer needs and tailor its approaches to enhance satisfaction and loyalty.

Kmeans

Once we have preprocessed the data, we can proceed with the clustering; the first method we will use is the kmeans algorithm. We start by determining the optimal number of clusters using the elbow method and the silhouette score. As shown in the plots below, both the elbow method and the silhouette score suggest that the optimal number of clusters is 3; we will then proceed with 3 clusters, as suggested by both the methods.

After feeding the data into the kmeans algorithm, we can visualize the clusters thanks to the Principal Component Analysis (PCA) that helps us in reducing the dimensionality of the data and visualizing the clusters in a 2D space. Hence, the plot below shows the customer segments identified by the kmeans algorithm.

K <- 3
set.seed(1)  
km_result <- kmeans(clustering_data, centers = K, nstart = 25, iter.max = 100)

The above scatter plot shows the customer segments identified by the kmeans algorithm. Each color represents a different cluster, and the points are plotted in a 2D space using the first two principal components. While the blue and the red clusters are well separated, indicating that the kmeans algorithm has successfully grouped customers based on their characteristics and behavior, the clusters exhibit considerable overlap, especially between the red and green clusters, indicating some shared characteristics among them.

## [1] 0.3016074

##   cluster size ave.sil.width
## 1       1 1249          0.32
## 2       2 1318          0.32
## 3       3  766          0.25

The silhouette plot for three clusters reveals varied cluster cohesion: Cluster 2 (Green) displays high silhouette widths, indicating strong cluster integrity and good separation from other clusters. In contrast, Clusters 1 (Red) and 3 (Blue) show lower and more varied silhouette widths, suggesting these clusters might be less well-defined and contain points that are close to the boundaries of other clusters. The average silhouette width across all clusters is 0.3, highlighting an overall moderate separation quality between the clusters. This suggests potential areas for improving the clustering approach or further investigation into the data characteristics.

Hierarchical Clustering

The second clustering method we will use is hierarchical clustering; we decided to use the same variables as for the kmeans algorithm in order to compare the results obtained with the two methods. We use the correlation-based method for hierarchical clustering which uses correlation coefficients as a measure of similarity between observations. We also consider different linkage methods to build the dendrogram and identify the optimal number of clusters: complete linkage, single linkage, average linkage, centroid linkage, and ward linkage.

dist_matrix <- dist(clustering_data)
set.seed(1)
h_complete <- hclust(dist_matrix, method = "complete")
h_single <- hclust(dist_matrix, method = "single")
h_average <- hclust(dist_matrix, method = "average")
h_centroid <- hclust(dist_matrix, method = "centroid")
h_ward <- hclust(dist_matrix, method = "ward.D2")

By plotting the dendorgrams for the different linkage methods, we can identify the optimal number of clusters. The Complete Linkage shows a more balanced structure, suggesting uniformity in cluster compactness. Single Linkage results in elongated clusters, prone to chaining effects, highlighting susceptibility to noise. Average Linkage offers a moderate clustering structure, more natural than Single or Complete Linkage. Centroid Linkage shows variations in cluster sizes, influenced by centroid positions which may shift due to outliers. Ward’s Method demonstrates very compact and well-defined clusters, effectively minimizing within-cluster variance, suggesting it as the most robust method for distinct and cohesive grouping in this dataset. As shown in the plots below, the ward linkage method in particular suggests that the optimal number of clusters is 3, which is consistent with the results obtained with the kmeans algorithm.

After identifying the optimal number of clusters, we can proceed by also checking the silouhette values achieved by the different linkage methods. When analyzed alongside dendrograms, high silhouette scores confirm the visual separation and compactness seen in methods like Ward’s, suggesting effective clustering. Conversely, low scores, particularly with methods like Single Linkage which shows elongated clusters, indicate issues such as chaining effects or potential misassignments. Moderate scores for Average or Complete Linkage reflect better clustering than Single but possibly with some overlaps.

##  complete    single   average  centroid   ward.D2 
## 0.1637051 0.4198898 0.2402278 0.3602106 0.2612211

Finally, we proceed with the hierarchical clustering using the ward linkage method and 3 clusters, as suggested by the silhouette scores. We then visualize the clusters using PCA, as we did for the kmeans algorithm. The PCA-based cluster visualization using Ward’s method clearly identifies three distinct customer groups in the dataset: a densely packed central cluster (blue), a broadly distributed cluster with some overlap (red), and a smaller, more distinct cluster (green). These clusters, represented on two principal components that likely capture the most significant variances in customer behavior such as usage, suggest specific strategies for targeted marketing, customer service enhancements, and product development. The graph validates the effectiveness of Ward’s method in hierarchical clustering by showing well-defined group separations, providing robust insights for strategic business decisions focused on tailored customer engagement and service offerings.

The comparison between the PCA plots of k-means clustering and hierarchical clustering using Ward’s method reveals distinct differences in clustering outcomes. K-means clustering exhibits greater overlap and less defined separation between clusters, particularly between the blue and red clusters, suggesting a less effective partitioning of the dataset. In contrast, hierarchical clustering demonstrates clearer and more distinct cluster boundaries, with notably more compact and homogeneous clusters. This indicates that hierarchical clustering with Ward’s method provides more precise and well-defined groupings, enhancing the interpretability and utility of the clustering results. Consequently, hierarchical clustering appears more suitable for this dataset, offering robust insights for targeted strategies and data-driven decisions.

Data Analysis for Business - Final Project

Edoardo Cocciò, Alexandra Tabarani, Marta Torella, Simone Filosofi

Standard Deviated Group

Task 1

EDA

Missing Values and Duplicates

Univariate Analysis

Hypothesis testing

Correlation analysis

Lower-dimensional models

Preprocessing and feature engineering

Splitting the data and scaling

Baseline Logistic Regression Model

AIC and BIC Model Selection

ROC Curves

LASSO and RIDGE REGULARIZATION

Other models

Decision Trees

Pruning the tree

Random Forest

eXtreme Gradient Boosting

Feature Importance

Main conclusions

Main conclusions

Task 2

Feature Engineering

Kmeans

Hierarchical Clustering